Semantic segmentation of images, assigning image pixels to one of several semantic classes, is one of the fundamental problems of computer vision. It is used in many applications, like image based modeling. In most current semantic segmentation methods, appearance features, like texture, color, are used to distinguish different categories. However, usually, three-dimensional information isn't utilized is these methods, as three-dimensional information cannot be recovered from a single image.
Nevertheless, three-dimensional information plays an important role in some semantic segmentation methods that target multiple view images or image sequences. The three-dimensional information in these methods is recovered by structure from motion (SFM) analysis, which takes an image sequence of a rigid (or static) object as the input and recovers the camera poses and a cloud of three-dimensional points. As three-dimensional information is important for many vision tasks, like scene understanding, great efforts have been devoted to the development of structure from motion (SFM) algorithms that can reconstruct three-dimensional information from multiple view images or image sequences.
However, although the structure from motion algorithms have made great progress on recovering three-dimensional information from images, the three-dimensional reconstruction with SFM suffers from a variety of limitations. For example, fast moving objects, like cars, cannot be reconstructed, and the density of reconstructed three-dimensional points is usually sparse.
Further, similar to the semantic segmentation of images, segmenting three-dimensional scan data into different semantic categories is a fundamental problem in the high-level tasks like scene understanding and planning of mobile robots. Most current semantic segmentation methods for three-dimensional scan data mainly use three-dimensional features, ignoring texture, color cues that are used in many image semantic segmentation methods.
The above-described deficiencies of conventional semantic segmentation techniques are merely intended to provide an overview of some of problems of current technology, and are not intended to be exhaustive. Other problems with the state of the art, and corresponding benefits of some of the various non-limiting embodiments described herein, may become further apparent upon review of the following detailed description.